1. Technical Field
This invention is directed toward a system and process, called Image-Based Surface Detail Transfer (IBSDT), for transferring geometric details from one surface in an image to another surface in another image with simple 2D image operations.
2. Background Art
Changing the appearance of an object by adding geometric details is desirable in many real world applications. For example, one may want to know what a wall might look like after adding some geometrical bumps on the wall, or one may want to know what a person might look like after adding/reducing wrinkles on his/her face, and so on. Adding geometric details to an object typically requires modeling both the object and the surface details. It is usually not trivial to build a 3D model for a real object. It is also typically tedious and labor intensive to model and create surface details with existing geometric modeling tools. Bump mapping [3] has been used as an alternative to adding geometrical details to an otherwise smooth object. But constructing visually interesting bump maps requires practice and artistic skills.
Computer vision techniques have been very helpful for modeling real world objects as well as the surface details. These techniques include laser scanner, steror algorithms, shape from lighting variation [8, 17], and shape from shading [10,9], among others. There are, however, many difficulties in the techniques used to model these real world images. Some of these techniques require specialized equipment. Many other techniques require at least two images for each object to be modeled, and it may be difficult to capture the high resolution geometrical details required for photo-realistic modeling robustly. Although shape from shading technique only requires a single image, this method usually requires detailed knowledge of the lighting condition and reflectance functions.
One use for changing the appearance of an object by adding geometric details is in the context of aging simulation. Aging simulation of human faces has applications in computer games, entertainment, cosmetics and virtual reality. Skin aging is a complex process that depends on multiple factors such as age, race, gender, health and even lifestyle. Anatomically, skin is attached to the underlying muscle by connective tissues and the attached end of a muscle is fixed to the skull. Facial appearance changes as the consequence of the gradual aging change of all of the facial components and the comprehensive interactions among these components. In spite of the difficulty of the problem, various techniques have been developed to analyze and synthesize facial aging effects. These methods can be roughly classified into three categories: model-based, image-based, and learning-based.
The model-based approach for facial aging effects is closely related to previous work on skin deformation simulation and skin texture synthesis. Wu et al. [18] proposed a three-layered Elastic Membrane Model for facial wrinkle simulation where “the deformation of skin is activated by the simulated muscle layer, constrained by the connective tissue layer and decided by a biomechanical model”. The skin model is computed with the aid of the feature points selected on the reconstructed face model. The wrinkles generated from the skin model are composed with real face images to produce the image of an aged face. An improved model was reported by Boissieux et al. [2], where the thickness and the mechanical properties of each skin layer are considered. This approach provides good insight into the nature of the aging process and can be used as guidelines in cosmetic and medical applications. In general, this approach requires 3D geometry information to perform physical simulation. The results are usually not as photorealistic as the image based approaches.
Boissieux et al. [2] developed an image-based method that uses eight generic masks generated from real photos of the aged people. Each mask contains quantitative information about the amount, shape and intensity of wrinkles with respect to gender, facial shape, and expression. To customize the face of a particular person, the wrinkle intensity (or depth) is computed and the mask is warped onto that face. The composition of the warped mask image and the image of the specific face forms the texture map of the final 3D model of the face. Because the generic masks contain mainly the wrinkle information, other morphological changes on the face due to aging cannot be reflected. An additional limitation of this method is that it cannot make an old face look younger. An additional-image based method was reported by Burson and Nancy [4]. It computes the differences of the aligned images of a young face and an old face. Given the image of another young face to be aged, the difference image is warped and added to this face to make it look older.
Learning-based approaches try to establish a statistical model for the aging process without understanding the underlying mechanisms. Lanitis et al. [11] suggested a linear face model of 15 parameters, obtained by Principal Component Analysis (PCA) on a set of normalized training examples. By using the same set of training data after sorting it according to age, they are also able to find a so-called aging function that relates the model parameters to the age. Choi [5] uses a PCA method to find the age related components for both skull and skin changes. By carefully choosing and normalizing the training examples, he is able to simulate the aging effect with the first principal components from both skull and skin data. The learning based approach is powerful because it does not rely on detail domain specific knowledge. It does, however, require a careful selection of the training data used.
It is noted that in the preceding paragraphs, as well as in the remainder of this specification, the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. A listing of the publications corresponding to each designator can be found at the end of the Detailed Description section.